System and method of adjusting audiovisual content to improve hearing

ABSTRACT

A system for adjusting audiovisual content having a server including a receiver for receiving a first content from a media source, a first circuitry configured to generate a second content based on the first content, the second content having multiple audio channels, a first transceiver for sending the second content to one or more clients, each client including a second transceiver for receiving the second content, and a second circuitry configured to generate a third content based on the second content and provide the third content to an audio device, wherein the second circuitry is configured to provide a graphical user interface to a user, the graphical user interface having a plurality of icons representing the user and the multiple audio channels, wherein movement of any of the plurality of icons will alter the second content.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.11/503,468, filed Aug. 11, 2006, the entire content of which isincorporated herein by reference, which claims priority to and thebenefit of U.S. Provisional Application No. 60/707,900 titled “EnhancedPersonal Audiovisual Technology” filed on Aug. 11, 2005, the entirecontent of which is incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to a system for assistingindividuals in the reception and understanding of information, includinginformation transmitted in the audio and visual domains.

BACKGROUND

Hearing is an important part of communication and can often be impairedfor varying reasons. In some instances, hearing is impaired due tophysical conditions of the human auditory system. In other instances,hearing is impaired due to the saturation of the audio domain by a largenumber of audio stimuli or distractions from non-audio stimuli. Oneexample is individuals immersed in noisy environments.

A variety of hearing impairments relate to physical conditions. Often anindividual may have difficulty hearing in only one ear, or moredifficulty hearing in one ear than the other. Some individuals may havedifficulty hearing out of either ear. Each person's hearing impairmentsare unique in terms of which ear is impaired and the degree to which itis impaired. Individual impairments to hearing can also be associatedwith particular frequencies of audio. In this regard, it is desirable tohave a system and method for adjusting audiovisual content to improvehearing.

SUMMARY

The present invention relates to a system and method for adjustingaudiovisual content. In one embodiment, invention relates to a systemfor adjusting audiovisual content having a server including a receiverfor receiving a first content from a media source, a first circuitryconfigured to generate a second content based on the first content, thesecond content having multiple audio channels, a first transceiver forsending the second content to one or more clients, each client includinga second transceiver for receiving the second content, and a secondcircuitry configured to generate a third content based on the secondcontent and provide the third content to an audio device, wherein thesecond circuitry is configured to provide a graphical user interface toa user, the graphical user interface having a plurality of iconsrepresenting the user and the multiple audio channels, wherein movementof any of the plurality of icons will alter the second content.

In another embodiment, the invention relates to a method of adjustingaudiovisual content including receiving a first signal having a firstcontent from a media source, generating a second content having multipleaudio channels based on the first content, providing a graphical userinterface having a plurality of icons representing a user and themultiple audio channels, wherein the second content is altered by movingany of the icons on the graphical user interface, generating a thirdsignal based on the second content, and providing the third signal to anaudio device.

In yet another embodiment, the invention relates to a system foradjusting audiovisual content having a server configured to receive afirst content from a media source, generate a second content havingmultiple audio channels based on the first content, and provide thesecond content to a client, where the client is configured to provide agraphical user interface to a user, the graphical user interface havinga plurality of icons representing the user and the multiple audiochannels, wherein movement of any of the plurality icons will alter thesecond content, generate a third content based on the second content,and provide the third content to an audio device.

In still yet another embodiment, the invention relates to a method ofadjusting audiovisual content including receiving a first signal havinga first content from a media source, generating a second content havingmultiple audio channels based on the first content, providing an inputconfigured to allow a user to alter the multiple audio channels,generating a third signal based on the second content, and providing thethird signal to an audio device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 a is a schematic block diagram of an audiovisual contentadjustment system in accordance with an embodiment of the presentinvention;

FIG. 1 b is a schematic block diagram of an audiovisual contentadjustment system in accordance with another embodiment of the presentinvention;

FIG. 1 c is a functional block diagram of an audio processing system inaccordance with the embodiment of FIG. 1 b;

FIG. 1 d is a functional block diagram of a video processing system inaccordance with the embodiment of FIG. 1 b;

FIG. 2 a is a schematic block diagram of a processor/transmitterinterface and several media sources in accordance with anotherembodiment of the present invention;

FIG. 2 b is a flowchart illustrating a process for operating aprocessor/transmitter in accordance with an embodiment of the presentinvention;

FIG. 2 c is a flowchart illustrating a process for operating acontroller/receiver in accordance with an embodiment of the presentinvention;

FIG. 2 d is a flowchart illustrating a process for operating a audioprocessing system in accordance an embodiment of the present invention;

FIG. 2 e is a flowchart illustrating a process for operating a videoprocessing system in accordance an embodiment of the present invention;

FIG. 2 f is a block diagram of a cell-phone comprising acontroller/receiver in accordance with an embodiment of the presentinvention;

FIG. 3 a is a table that lists characteristics of a graphical userinterface in accordance with an embodiment of the present invention;

FIGS. 3 b-3 i are illustrations of a graphical user interface foradjusting audio content in accordance with an embodiment of the presentinvention;

FIG. 4 a is a table that lists characteristics of a graphical userinterface in accordance with an embodiment of the present invention;

FIG. 4 b is a screen shot of a video delivery system in accordance withan embodiment of the present invention;

FIG. 4 c is a screen shot of a graphical user interface for adjustingvideo content in a video delivery system in accordance with anembodiment of the present invention;

FIG. 5 a illustrates a general message packet format in accordance withan embodiment of the present invention;

FIG. 6 a is a flowchart illustrating a process for operating aprocessor/transmitter in accordance with an embodiment of the presentinvention;

FIG. 6 b is a flowchart illustrating a process for operating acontroller/receiver in accordance with an embodiment of the presentinvention;

FIG. 6 c illustrates the format of a metadata packet in accordance withan embodiment of the present invention;

FIG. 6 d illustrates the format of a audio packet in accordance with anembodiment of the present invention;

FIG. 6 e illustrates the format of a video packet in accordance with anembodiment of the present invention;

FIG. 6 f illustrates the format of an acknowledgement packet inaccordance with an embodiment of the present invention;

FIG. 6 g illustrates the format of an NACK packet in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION

Referring now to the drawings, audiovisual content adjustment systems inaccordance with embodiments of the present invention are illustrated.Audiovisual content adjustment systems can allow a user to individuallyand graphically customize the channels or subcomponents of audio contentand video content from a media source to suit the user's preferences. Insome cases, the user's preferences will be tailored to overcomeimpairments to the user's hearing. The user can often express hispreferences visually with the use of a graphical user interface. Theuser can adjust the audio content using a graphical user interfacedepicting a room or theater with a multi-channel speaker system, as inFIGS. 3 b-3 i.

Often a multichannel audio system will include six speakers comprisingcenter, right, left, right surround, and left surround speakers, as wellas a special effects speaker, which is typically a subwoofer. By movingicons representing speakers and/or the user, the user can control theway the audio content sounds when played on an audio delivery system,like a pair of headphones. For example, if the center speaker icon ismoved closer to the icon depicting the user, then the volume of thecenter speaker relative to the other speakers will be increased whenplayed on the user's headphones. The increase in volume can be inproportion to the distance that the center speaker icon is moved.

Video content from the media source can supplement the audio content byproviding close-captioning text representing the audio content deliveredto the audio delivery system. Characteristics, such as size, color andtransparency of the text, describing how the video content is presentedto the user can be customized using a graphical user interface. In oneembodiment, the video content and graphical user interface is presentedto the user by a video delivery system such as a heads-up display.

A system for transporting audio content, video content including theclose-captioning text and any other appropriate content to the user froma media source can include a communication channel capable oftransporting digital message packets, as in FIGS. 1 a and 1 b. The audioand video content can be received from various analog and digital mediasources and then separated using an audiovisual server. The audiovisualserver can send the audio and video content separately in packet formatto an audiovisual client. In such a case, the audiovisual clientreceives and identifies packets of different formats. Packets formatscan include audio, video and metadata packets. Metadata packets describethe content of data in the other packets. For example, a metadata packetmight describe the author, title, and/or year recorded of a filmcurrently being played with the audiovisual content system.

A schematic block diagram of an audiovisual content adjustment system inaccordance with an embodiment of the present invention is shown in FIG.1 a. The audiovisual content adjustment system 10 includes an analogmedia source 12, a digital media source 14, an audiovisual server 16, acommunication channel 17, an audiovisual client 18, an audio deliverysystem 20 and a video delivery system 22. The analog media source 12 anddigital media source 14 are connected to the audiovisual server 16. Theaudiovisual server 16 is connected to the audiovisual client 18 via thecommunication channel 17. The audiovisual client 18 is connected to theaudio delivery system 20 and the video delivery system 22.

In operation, the audiovisual server 16 can receive audio content, videocontent, and/or other appropriate content from the analog media source12 and/or the digital media source 14. If the media source is analog,then the analog audio and/or video content can be converted into adigital form. The digital audio and/or video content can then besegmented into packets such that some packets carry audio content andsome packets carry video content while other packets carry metadata orother appropriate content. The metadata can be thought of as informationdescribing other content, such as information describing thecharacteristics of audio content.

The audiovisual server 16 can then transport the packets to theaudiovisual client 18 over the communication channel 17. The audiovisualclient 18 can receive the packets and separate them according to type.The audiovisual client can include a graphical user interface capable ofdetermining user preferences for modifying audio content. The packets ofaudio content can be processed according to user preferences determinedusing the graphical user interfaces and delivered to an audio deliverysystem 20.

The audio delivery system 20 can allow a user to listen to the audiocontent with, for example, a pair of headphones. The packets of videocontent can be processed according to user preferences determined usingthe graphical user interfaces and delivered to a video delivery system22. The packets of metadata can be processed and also delivered to thevideo delivery system 22.

In many embodiments, a user of the audiovisual adjustment system canmake changes to the way audio content is presented on the audio deliverysystem. In several embodiments, a user of the audiovisual adjustmentsystem can make changes to the way the video content, often in the formof close-captioning text, is presented on the video delivery system.

In many embodiments, the user makes changes to the presentation of audiocontent using a graphical user interface. In some embodiments, the usermakes changes the presentation of video content and/or metadata using agraphical user interface.

In one embodiment, communication via the communication channel isachieved using an analog signal over a wireless system. In this case,audio and video content in analog form is transported by the audiovisualserver to the audiovisual client.

A schematic block diagram of an audiovisual content adjustment system inaccordance with an embodiment of the present invention is shown in FIG.1 b. The audiovisual content adjustment system 10′ includes an analogmedia source 12′, a digital media source 14′, a processor/transmitter16′, a communication channel 17′, a controller/receiver 18′, an audiodelivery system 20′ and a video delivery system 22′. The analog mediasource 12′ and digital media source 14′ are connected to theprocessor/transmitter 16′. The processor/transmitter 16′ is connected tothe controller/receiver 18′ via the communication channel 17′. Thecontroller/receiver 18′ is connected to the audio delivery system 20′and the video delivery system 22′.

In operation, the processor/transmitter 16′ can act as an audiovisualserver and the controller/receiver 18′ can act as an audiovisual client.The processor/transmitter 16′ can receive audio and/or video contentfrom the analog media source 12′ in analog form. Theprocessor/transmitter 16′ can receive audio and/or video content fromthe digital media source 12′ in digital form.

Analog media sources can include music playback systems, film playbacksystems, audio playback systems, public address systems, broadcast mediasystems and other appropriate analog media sources. Digital mediasources can include music playback systems such as compact disc (CD)players, film playback systems such as digital video disc (DVD) playersor digital film processors, audio playback systems, and otherappropriate digital media sources.

The processor/transmitter 16′ can receive, process, and transmit theaudio and/or video content over the communication channel 17′ to thecontroller/receiver 18′. The controller/receiver 18′ can receive,process and distribute the audio and/or video content received from theprocessor/transmitter 16′ to both the audio delivery system 20′ andvideo delivery system 22′.

The processor/transmitter 16′ includes an analog to digital converter(ADC) 24, a digital input processor 26, a processor 28, a memory 29, anda transceiver 30. The processor 28 is connected to the ADC 24, thedigital input processor 26, the memory 29 and the transceiver 30. Thetransceiver 30 is connected to the communication channel 17′ and thememory 29. The ADC 24 is connected to the analog media source 12′. Thedigital input processor 26 is connected to the digital media source 14′.

In operation, the ADC 24 converts analog media received from the analogmedia source 12′ to digital media. The digital input processor 26processes the media from the digital media source into a format suitablefor the processor 28 and for storing the media in the memory 29. In oneembodiment, digital media sources can include digital audio, digitaltext corresponding to the digital audio, and digital time codes. Theprocessor assembles the digital audio and/or video content into packets.The packets may include packets of audio content, video contentincluding close-captioning text, and metadata.

In assembling the packets of audio content, the processor multiplexesmultiple channels of audio to comprise the audio data to be contained inthe audio packet. Thus, the audio data contains portions of data eachattributable to one of the audio channels. In one embodiment, theprocessor multiplexes six channels of audio including a center, left,right, left surround, right surround, and special (often a subwoofer)channel. In other embodiments, the number of channels can be more thanor less than six.

The processor sends the packets to the transceiver 30 to be transmittedon the communication channel 17′. In one embodiment, the transceiver 30has its own memory (not shown) or uses the memory 29 to buffer packetsto be transmitted on the communication channel 17′.

The controller/receiver 18′ includes a transceiver 32, a memory 33, aprocessor 34, an audio communication channel 35, an audio processingsystem 36, a video communication channel 37 and a video processingsystem 38. The transceiver 32 is connected to the communication channel17′ and the memory 33. The processor 34 is connected to the transceiver32, the memory 33, the audio processing system 36 via the connection 35and the video processing system 38 via the connection 37. The audioprocessing system 36 is connected to the audio delivery system 20′. Thevideo processing system 38 is connected to the video delivery system22′. The controller/receiver can include other components thatfacilitate the receipt, processing and conversion of packets. Forexample, additional memory or processors can be used.

In operation, the transceiver 32 receives packets from the communicationchannel 17′ and sends them to the processor 34. In one embodiment, thetransceiver 32 sends data to the processor/transmitter 16′. For example,the transceiver can send an acknowledgement of receipt of a packet. Theprocessor or transceiver 32 can store packets in the memory 33 as neededto buffer the received audio and/or video content. The processor 34identifies the type of packet received and sends it to the appropriatesystem. The processor 34 sends packets of audio content to the audioprocessing system and packets of video and/or metadata content to thevideo processing system 38. The audio processing system converts thepackets of audio content into a form suitable for the audio deliverysystem. In one embodiment, the audio delivery system is a pair ofheadphones. The video processing system converts the packets of videocontent and/or metadata into a form suitable for the video deliverysystem. In one embodiment, the video delivery system is a heads-updisplay system which is capable of displaying text within the field ofvision of a user.

In one embodiment, the processor/transmitter 16′ can be implementedusing a combination of processing elements such as digital signalprocessors (DSPs), programmable logic, discrete logic components and/orother circuitry capable of sharing information. In one embodiment, thecontroller/receiver 16′ can be implemented using a combination ofprocessing elements such as digital signal processors (DSPs),programmable logic, discrete logic components and/or other circuitrycapable of sharing information.

In several embodiments, the communication channel 17′ can be a wirelesscommunication channel. In several embodiments, the wirelesscommunication channel can use radio frequency (RF) bands to communicate.In one embodiment, the wireless protocol can comply with the IEEE 802.11or related family of wireless protocols including Wi-Fi. In anotherembodiment, the wireless protocol can comply with the Bluetooth, or IEEE802.15.1, standard. The wireless communication channel can use anynumber of appropriate protocols for sending packets. In one embodiment,the frequency band used for wireless communication can includecell-phone bands such as 900 MHz. In other embodiments, other cell-phonebands can be used.

In one embodiment, the communication channel can be a wiredcommunication channel. The wired communication channel can be a copperwire, twisted pair, fiber or other appropriate medium.

A functional block diagram of an audio processing system in accordancewith an embodiment of the present invention is shown in FIG. 1 c. Theaudio processing system 36 includes a signal from the processor 35, ademultiplexer 40, signal processing 42, digital filters 44, channelprocessing 46, a routing matrix 48, a configuration control block 50, auser interface 54, coefficient processing 56 and a audio delivery system58. The demultiplexer is connected to the signal from the processor 35and the signal processing block 42. The signal processing block 42 isconnected to the digital filters 44, the coefficient processing block 56and the channel processing block 46. The coefficient processing block 56is connected to the user interface 54 and the channel processing block46. The channel processing block 46 is connected to the routing matrix48 and the audio delivery system 58.

In operation, the demultiplexer 40 can receive packets of audio datafrom the signal from the processor 35 and separate the multiple channelsof audio that have been multiplexed into the audio packet (i.e.demultiplex the multiple channels of audio content). The demultiplexer40 can send the demultiplexed audio content (now comprising multiplechannels) to the signal processing block 42. The signal processing block42 can apply digital filters 44 and coefficient processing 56 to processthe multiple channels of audio content according to the preferences ofthe user 52. The coefficient processing 56 takes input from the userinterface 54 which can receive input from a user 52. The user 52 canprovide input indicating any changes he/she would prefer in thecharacteristics of the audio content. In one embodiment, userpreferences can include the volume of any channel or positioninformation relating to the location of each channel relative to theother channels.

The coefficient processing block 56 develops the parameters necessary tomodify the audio content as indicated by the user and sends thoseparameters to both the signal processing block 42 and the channelprocessing block 46. The signal processing block 42 modifies the audiocontent according to the user parameters from the coefficientprocessing. In one embodiment, the audio configuration preferences arestored (i.e. default settings) and can be recalled after the user isidentified using the user interface.

In modifying the audio content, the signal processing block 42 mayincrease or decrease the volume of any of the multiple audio channelsaccording to the preferences of the user. After the audio content isfiltered per the user's preferences, the channel processing block 46uses the routing matrix 48 and user preferences to combine the multiplechannels of audio content into two channels, a left channel and a rightchannel. The channel processing block adjusts the volumes of the leftand right channels according the user preferences and/or defaultsettings. The left and right channels can be converted to analog format.An audio delivery system 58 receives the right and left channels fromthe channel processing block 46.

In one embodiment, the audio delivery system is a pair of headphones. Inanother embodiment, the audio delivery system can be one or morespeakers. In one embodiment, the processing in the audio processingsystem 36 can be performed by a single processor, such as thecontroller/receiver processor 34. In other embodiments, the audioprocessing system 36 can be implemented using a combination ofprocessing elements such as digital signal processors (DSPs),programmable logic, discrete logic components and/or other circuitrycapable of sharing information.

A functional block diagram of an video processing system in accordancewith an embodiment of the present invention is shown in FIG. 1 d. Thevideo processing system 38 includes a signal from the processor 37, ademultiplexer 60, a signal conditioning block 62, presentationcoefficients 64, a video processor 66, a temporary storage 68, aconfiguration control block 70, a user interface 74, coefficientprocessing 76 and a video delivery system 78. The demultiplexer 60 isconnected to the signal from the processor 37 and the signalconditioning block 62. The signal conditioning block 62 is connected tothe presentation coefficients 64, the coefficient processing block 76and the video processor 66. The coefficient processing block 76 isconnected to the user interface 74 and the video processor 66. The videoprocessor is connected to the temporary storage 68 and the videodelivery system 78.

In operation, the demultiplexer 60 can receive packets of video datafrom the signal from the processor 37 and separate the text data andtiming data that have been multiplexed into the video packet (i.e.demultiplexes the text and timing data). The demultiplexer 60 can sendthe demultiplexed video content to the signal conditioning block 62. Thesignal processing block 62 can apply presentation coefficients 64 andcoefficient processing 76 to process the demultiplexed video content.The coefficient processing 76 takes input from the user interface 74which can receive input from a user 72. The user 72 can provide inputindicating the changes he/she would prefer in the characteristics of thevideo content.

The coefficient processing block 76 develops the parameters necessary tomodify the video content as indicated by the user and sends thoseparameters to both the signal conditioning block 62 and the videoprocessor 66. The signal conditioning block 62 modifies the videocontent according to the user parameters from the coefficient processing76. In one embodiment, the video configuration preferences are stored(i.e. default settings) and can be recalled after the user is identifiedusing the user interface.

In modifying the video content, the signal conditioning block 62 canmodify the size, color, translucence and any other appropriate videocontent characteristics according to the preferences of the user. Afterthe video content is modified according to the user's preferences, thevideo processor 66 uses temporary storage 68 and the user's preferencesto present video data, at the time specified by time codes and convertedto a analog video signal, to the video delivery system 78.

In one embodiment, the video delivery system is a lightweighthead-mounted visual projection system suitable for use with thisinvention such as the monochrome Nomad system by Microvision, Inc ofBothell, Wash. in which the visual image (i.e., text and graphic images)is projected directly to the back of the retina. The Nomad system is ascan-beamed display system that focuses low-power lasers into the foveaarea of the retina allowing for images to be “laid” over the viewer'sview of real objects. The Nomad system is monochrome; the default textcolor is red, which has proven a suitable font color in automotiveapplications.

In other embodiments, the video delivery system can be any systemsuitable for displaying text. In one embodiment, the processing in thevideo processing system 38 can be performed by a single processor, suchas the controller/receiver processor 34. In one embodiment, the videoprocessing system 38 can be implemented using a combination ofprocessing elements such as digital signal processors (DSPs),programmable logic, discrete logic components and/or other circuitrycapable of sharing information.

A schematic block diagram of a processor/transmitter interface andseveral media sources in accordance with an embodiment of the presentinvention are shown in FIG. 2 a. The system 80 includes a digital soundprocessor 82, communication channels 84, 86, 88, 91, 95, 97, a DVDsource 90, a film projector 92, a time code generator 94, an analogpickup 96, a processor/transmitter 16″ and a communication channel 17″.

The digital sound processor 82 is connected to the film projector 92 andthe processor/transmitter 16″ by channels 88, 86 and 84. In oneembodiment, the digital sound processor is a Sony DFP-D3000 made by SonyCorp. of Tokyo, Japan. The DVD source 90 is connected to theprocessor/transmitter 16″ via channel 91. The time code generator 94 isconnected to the film projector 92 and to the processor/transmitter 16″via channel 95. The analog pickup 96 is connected to the film projector92 and to the processor/transmitter 16″ via channel 97.

The channel 84 is a single digital signal and can transport multiplexedtime code data. The channel 86 is a single digital channel and cantransport error data and time code data. The channel 88 can be comprisea number of distinct audio channels. In one embodiment, the channel 88comprises six audio channels. In other embodiments, the channel 88comprises more than or less than six channels. The channel 91 caninclude digital data comprising video, audio, text, metadata, and/ortime code data from the DVD 90. The time code generator can generate adigital signal that provides information for synchronizing the analogaudio content with the associated video content, includingclose-captioning text. In one embodiment, the output of the filmprojector includes both analog time code information and analog audio.The channel 95 can transport a digital time code signal. The channel 97can transport multiple analog audio channels. In one embodiment, thechannel transports 97 four analog audio channels. In other embodiments,the channel 97 transports more than or less than four audio channels. Inone embodiment, the processor/transmitter 16″ operates as described inthe discussion of FIG. 1 b.

A flowchart illustrating a process for operating a processor/transmitter(P/T) in accordance with an embodiment of the present invention is shownin FIG. 2 b. The P/T receives audio data (108). If the data is analog(110), then the P/T extracts (112) audio signals. In one embodiment, therate of extraction is 44.1 MHz per channel. If the data is from adigital sound processor (DSP) (114), then the P/T extracts digital timecode (106) from the digital time code from the DSP 104. If the data isnot from a DSP, then the P/T extracts (102) digital time code from thedigital time code from the time code generator (100). If the audio datais not analog (110) (i.e. audio data is digital), then the P/T extracts(116) the digital signal. The P/T demultiplexes (118) the digital datainto multiple channels and time codes. If digital text is available(124), then the P/T extracts (122) the text data and time code andcreates (123) digital text packets including embedded time codes.

The P/T then creates (126) digital audio packets including embedded timecodes. The P/T then generates (128) a message stream that includes thedigital audio packets and embedded text packets based on time codes. Inone embodiment, audio and text are synchronized by the time code andtext packets are embedded into the stream based on the time code. In oneembodiment, text can be displayed on the video display device for atleast ten seconds and thus the message stream can be dominated by audiopackets. The P/T then embeds 130 metadata packets every five seconds. Inother embodiments, the P/T can embed metadata more often or less oftenthan five seconds. The P/T then stores the packets in a message cache(132) to be transmitted. If the cache is empty (134), then the P/T keepschecking to see if the message cache is empty. If the cache is notempty, then the P/T transmits (136) the next packet.

In one embodiment, packets are transmitted at a rate to ensure adequateplayback of audio content. In one embodiment, packets can be broadcastto all controller/receivers (i.e. without individual addressing). Inother embodiments, other suitable processes can be used to extractmultiple channels of audio, time code and accompanying text while alsopackaging and transmitting the information.

A flowchart illustrating a process for operating a controller/receiver(C/R) in accordance with an embodiment of the present invention is shownin FIG. 2 c. The C/R is provided a digital signal stream (138). The C/Rtemporarily stores incoming packets in a data cache (140). The C/Rextracts (142) the next packet from the cache and extracts (144) theidentifier of the intended receiver from the packet. The general messagepacket format in accordance with one embodiment of the present inventionis shown in FIG. 5 a. The identifier of the receiver occupies bytes 5-10of the general message packet.

Returning to the flowchart of FIG. 2 c, if the identifier indicates thatthe packet is not a broadcast packet (146), then the C/R checks if theunique identifier of the C/R matches (148) the packet identifier. If theunique identifier of the C/R does not match the packet identifier, thenthe C/R extracts (142) the next packet. If either the packet is abroadcast packet or matches the unique C/R identifier, then the C/Rparses (150) the packet to determine the type of packet (i.e. digitalaudio, digital text, metadata, etc.). If the packet/message headeridentifier (byte 1 of the packet) indicates that the packet is an audiopacket (152), then the C/R extracts (154) time code information,extracts (156) the audio packet data and extracts (158) the controlinformation. The C/R then demultiplexes (160) individual channels ofaudio content. The C/R then stores (162) the multiple audio channel dataand time code data, which can be provided to the audio processing system36.

If the packet is not an audio packet (152), then the C/R checks to seeif the packet is a metadata packet (164). If the packet is a metadatapacket, the C/R extracts (166) the metadata (often in the form of filmtitle and other control information) and stores (168) it. If the packetis not a metadata packet, then the C/R checks to see if the packet is avideo packet (170). If the packet is not a video packet, then the C/Rextracts (142) the next packet. If the packet is a video packet, thenthe C/R extracts (172) time code information, extracts (174) the videopacket data and extracts (176) the control information. The C/R thendemultiplexes (178) the video content (text) and time code data. The C/Rthen stores (180) the text and time code data, which can be provided tothe audio processing system 36.

A flowchart illustrating a process for operating a audio processingsystem in accordance with an embodiment of the present invention isshown in FIG. 2 d. Multiple channel audio data and time code data (184),user audio configuration parameters (182), and default parameters (210)are provided to the process. The processor checks to see if gain/cutparameters are available (186) from the user audio configurationparameters.

If gain/cut parameters are not available, then the processor extracts(212) the default gain/cut parameters from the default parameters. Ifthe gain/cut parameters are available, then the processor extracts (188)the gain/cut parameters from the user audio configuration parameters.The processor then checks to see if the location parameters areavailable (190). If the location parameters are not available, then theprocessor extracts (214) the default location/position parameters fromthe default parameters. If the location parameters are available, thenthe processor extracts (192) the user defined location/positionparameters.

The processor then applies (194) the digital filter to all the audiochannels. The processor then extracts (196) a routing matrix from thedefault parameters. In one embodiment, the routing matrix can indicatethe way in which a system with more than two channels is to be convertedto a system with only two channels (i.e. right and left audio channels).After extracting the routing matrix, the processor generates (198) leftand right channels. The processor then checks to see if the user defined(200) left and/or right gain or cut parameters. If the user did notdefine left and/or right gain or cut parameters, then the left and/orright gain or cut parameters are extracted (216) from the defaultparameters.

If the user did define left and/or right gain or cut parameters, thenthe left and/or right gain or cut parameters are extracted (202) fromthe user defined parameters. The processor then applies (204) a digitalfilter to the left and right channels. With the use of the digitalfilter, the processor can modify the left and right channels accordingto the specified parameters. The process is provided data from the timecode cache (218). The processor uses the time code cache in converting(206) the left and right channels to analog signals. The analog signalscan then be sent to headphones or earphones (208).

A flowchart illustrating a process for operating a video processingsystem in accordance with an embodiment of the present invention isshown in FIG. 2 e. Default parameters (210′), video and/or text data andtime code data (224), user text configuration parameters (236), and timecode cache data (218′) are provided to the process. The processor checksthe video/text and time code data to see if text data is available(226). If text data is not available, then the processor checks againfor text data within the video/text and time code data. If text data isavailable, then the processor checks to see if font parameters (color,size, etc.) are available (228) from the user text configurationparameters (236). If the font parameters are not available from theuser, the processor extracts (220) default font parameters from thedefault parameters (210′). If the font parameters are available from theuser, the processor extracts (230) the user defined font parameters. Theprocessor then checks to see if the location/position parameters areavailable (232). If the location/position parameters are not available,then the processor extracts (222) the default location/positionparameters from the default parameters (210′). If the location/positionparameters are available, then the processor extracts (234) the userdefined position parameters. The processor then stores (238) the textdata and parameters the temporary text and parameter cache (246). Theprocessor then check to see if the time code of the video/text data isequal to or less than the audio time code (240) retrieved from the timecode cache (218′). If the time code of the video/text data is not equalto or less than the audio time code, then the processor keeps checkinguntil the time code of the video/text data is equal to or less than theaudio time code. Once the time code of the video/text data is equal toor less than the audio time code, the processor then converts (242) thedigital video/text signals to analog and sends (244) them to the videodelivery system.

A block diagram of a cell-phone comprising a controller/receiver inaccordance with an embodiment of the present invention is shown in FIG.2 f. The controller/receiver/cell-phone 300 includes traditionalcell-phone components 301, 303, 305, 307, 309, 311, 313, 315, 317, 319,321 and controller/receiver components 302, 304, 306, 308, 310, 312,314, 316, 318, and 320. A skilled artisan would appreciate that thetraditional cell-phone components can be connected and be operated inthe ways known to those skilled in the art to provide functions typicalof a cell-phone. The controller/receiver components can be operated inthe ways discussed herein.

The controller/receiver components are indicated with dashed lines. Thecontroller/receiver/cell-phone 300 receives input from antennas capableof receiving broadcast frequencies and/or cell-phone frequencies. Thecontroller/receiver/cell-phone 300 can include a visual display device(not shown) or provide an output that is configured to provideinformation to an external video display device 22′. Thecontroller/receiver/cell-phone 300 can also provide output(s) that areconfigured to provide audio content for stereo headphones 20″ or stereospeakers 20″ or mono headphones 20′. The mode switch 310 can place thecontroller/receiver/cell-phone 300 in a mode for traditional cell-phoneoperation or a mode for operation as a controller/receiver.

The touch screen control can provide a graphical user interface to theuser of the controller/receiver/cell-phone.

Illustrations of a graphical user interfaces (GUIs) for adjusting audiocontent in accordance with an embodiment of the present invention areshown in FIGS. 3 b-3 i. FIG. 3 b shows a GUI having speaker locationsfor typical theater environment with multiple channel audio inaccordance with an embodiment of the present invention. The graphicaluser interface 54′ includes a room or theater 328, a left speaker icon330, a center speaker icon 332, a right speaker icon 334, a leftsurround speaker icons 336, a right surround speaker icons 338, asubwoofer icon 340 and a user icon 326. The left surround speaker icons336 include one or more speaker icons. The right surround speaker icons338 include one or more speaker icons.

FIG. 3 a is a table that lists characteristics of a graphical userinterface in accordance with an embodiment of the present invention.Some of the possible operations are listed with the correspondingmodifications and comments.

FIG. 3 c shows the functional location of audio channels from a user'sperspective in a typical theater environment, in accordance with oneembodiment of the present invention. The GUI 54″ includes a room 328′, adialog speaker icon 342, a left channel icon 336′, a right channel icon338′ and a sub-woofer icon 340′. FIG. 3 d shows a GUI after the user(not shown) has moved the user icon 326 closer to the center speakericon 332, in accordance with one embodiment of the present invention.The system can respond by increasing the center, left and right channelvolume and decreasing the left and right surround volume. The magnitudeof the increase or decrease (cut/gain) of volume corresponds to thedistance the icon (user icon or speaker icon) is moved.

FIG. 3 e shows a GUI after the user (not shown) has moved the centerspeaker icon 332 and sub-woofer icon 340 closer to the user icon 326, inaccordance with one embodiment of the present invention. In oneembodiment, the system can respond by increasing the volume of thecenter and sub-woofer channels and decreasing the volume of theremaining channels. In another embodiment, the system can respond byincreasing the volume of the center and sub-woofer channels whilemaintaining the volume of the remaining channels.

The user can change or adjust the audio content by placing icons in anyarrangement he desires. Moving just a few icons might constitute a minoralteration while moving several icons can be a major alteration. Theeffect of making an alteration can have a pronounced effect on thevolume or just a minor effect.

FIG. 3 f shows a GUI with the addition of a left volume icon 344 and aright volume icon 346, in accordance with one embodiment of the presentinvention. By clicking on the left or right volume icons, the user canadjust the volume (gain/cut) of the left and right composite signals.

FIG. 3 g shows a GUI after the user (not shown) has moved the centerspeaker icon 332 to the left side of the user icon 326, in accordancewith one embodiment of the present invention. In response, the systemcan adjust the volume such that the center channel is now perceived bythe user as originating from the left of the user.

FIG. 3 h shows a GUI with the addition of a up control icon 348 and adown control icon 350. The up control icon 348 and down control icon 350can enable the user to change whether a particular channel will bereproduced such that it is perceived as being higher or lower than thedefault setting. The default setting (height) may vary depending on thechannel. For example, FIG. 3 i shows a GUI after the user (not shown)has elevated the center channel 332″ and lowered one of the leftsurround speakers 336″, in accordance with one embodiment of the presentinvention.

A graphical user interface can also be used to change thecharacteristics of video/visual data (text) that is delivered to theuser. FIG. 4 a is a table that lists the characteristics of a graphicaluser interface that can be used to adjust the video/visual content inaccordance with an embodiment of the present invention.

A screen shot of a video delivery system in accordance with anembodiment of the present invention is shown in FIG. 4 b. A screen shotof a graphical user interface for adjusting video content in a videodelivery system 74′ in accordance with an embodiment of the presentinvention is shown in FIG. 4 c. The user (not shown) can adjust the fontsize 356, font color 358, text transparency (trans) 360. The user canadjust the placement of the text within the user visual field 352 bymoving the text field box 354.

Returning to FIG. 1 b, the communication protocol between theprocessor/transmitter and controller/receiver involves the exchange ofpackets containing metadata, audio, or text data. Control bytes can beused to identify the content of the packet. Each packet/message createdcan start with a header identifier, for example, total number of bytesin the packet/message, identifier to receive the packet (blank ifbroadcast), command and sub-command identifiers (message specific),read/write identifier (message specific), audio data or metadata, and atwo-byte checksum calculation.

The general message packet format in accordance with an embodiment ofthe present invention is illustrated in FIG. 5 a. The checksum can becalculated by starting at the packet/message header (first byte) andsumming all bytes up to the last data byte. The checksum can be storedin 8-bit binary format. The 2's compliment code of the least significantbyte of the sum total is used in the checksum. The checksum isrepresented in ASCII HEX form. For example, in the following commandmessage, with a 2-byte “number of bytes in message” filed, the first 5bytes are summed resulting in the 8-bit code.

The 2's compliment of BBH is 45H and is stored in the last two bytes ofthe message (bytes 6 and 7) in ASCII HEX.

A flowchart illustrating a process for operating a processor/transmitter(audiovisual server) in accordance with an embodiment of the presentinvention is shown in FIG. 6 a. The process is provided with audio data(370), text data (378) and metadata (390). The process checks to see ifaudio data is available (372). If audio data is not available, then theprocessor keeps checking to see if audio data is available (372). Ifaudio data is available, then the processor creates an audio packet(374) and sends (376) the packet. In one embodiment, the audio packetfollows the format set out in FIG. 6 d.

The processor checks to see if text data is available (380). If textdata is not available, then the processor keeps checking to see if textdata is available (380). If text data is available, then the processorcreates (382) a text packet. In one embodiment, the text packet followsthe format set out in FIG. 6 e. The processor then checks to see if thetext time code is less than or equal to (384) the audio time code.

If the text time code is not less than or equal to the audio time code,then the processor returns to checking if text data is available (380).If the text time code is less than or equal to the audio time code, thenthe processor checks to see if the five second time has expired (386).If the five second timer has not expired, the processor returns tochecking if the text time code is less than or equal to the audio timecode (384). If the five second timer has expired, then the processorsets (388) the timer and sends (376) the packet.

The processor creates (392) a metadata packet from the metadata (390)provided. In one embodiment, the metadata packet follows the format setout in FIG. 6 c. The processor then checks to see if the five secondtime has expired (394). If the five second timer has not expired, theprocessor returns to checking if the five second time has expired (394).If the five second timer has expired, then the processor sets (396) thetimer and sends (376) the packet.

A flowchart illustrating a process for operating a controller/receiver(audiovisual client) in accordance with an embodiment of the presentinvention is shown in FIG. 6 b. The process receives data packets (400).The processor checks to see if the data packet is (402) a data packetfor the controller/receiver (C/R) or audiovisual client.

If the data packet is not meant for the C/R, then the processor returnsto checking to see if the next data packet is meant for the C/R (402).If the data packet is meant for the C/R, then the processor determinesif the data packet is a metadata packet (404). If the data packet is ametadata packet, then the processor updates (406) configurationinformation. In one embodiment, the configuration information caninclude the number of audio channels, the film title, text messages, orother pertinent information. If the data packet is not a metadatapacket, then the processor checks to see if the data packet comprisesaudio data (408).

If the data packet comprises audio data, then the processor processes(410) the audio data and returns to checking to see if the next datapacket is meant for the C/R (402). If the data packet does not compriseaudio data, then the processor checks to see if the data packetcomprises (412) text data. If the data packet comprises text data, thenthe processor processes the text/video data and returns to checking tosee if the next data packet is meant for the C/R (402).

If the data packet does not comprise text data, then the processorchecks for any future use packets and processes them accordingly. Futureuse packets by definition have not been specified, but any relevant andsuitable information can be included in a future use packet to bedefined in the future and as needed. After processing any futurepackets, the processor can return to checking to see if the next datapacket is meant for the C/R (402).

In the communication protocol between the processor/transmitter (server)and controller/receiver (client), general command messages can be sentby the server to the client with specific command message information.Acknowledgment (ACK) messages can be sent by the client to the server toindicate the successful receipt of the command message sent by theserver. Each command message has a specific ACK message. FIG. 6 fillustrates the format of an acknowledgement packet in accordance withan embodiment of the present invention.

Non-acknowledgment (NACK) commands can be sent when the client is notable to service the incoming request (i.e., a command message was sentto the server). In one embodiment, there two types of NACK messages canbe used: generic NACK messages and command specific NACK messages.Generic NACK messages generally relate to communication problems thateffect all command message types. Such communication failures caninclude checksum errors and processor time out. Error conditionsspecific to the command message sent to the processor can require anon-generic NACK message. FIG. 6 g illustrates the format of an NACKpacket in accordance with an embodiment of the present invention.

In one embodiment, a person with impaired hearing operates a system ofadjusting audiovisual content in a theater. The theater has anaudiovisual server configured to receive digital media from a DVDplayer. The person or user has a cell-phone or PDA integrated with anaudiovisual client or controller/receiver (“integrated client”). Incomparison to a standard cell-phone or PDA, the integrated clientincludes additional components or is implemented completely in softwarethat runs on the integrated client.

The theater begins playing a movie with the DVD player. The audiovisualserver receives digital content from the DVD player, multiplexesportions of the digital content, places the digital content into packetsand transmits the packets in a broadcast mode within the theater. Inbroadcast mode, multiple users can receive and customize the audiovisualcontent.

The user places the integrated client in a mode to receive packets fromthe audiovisual server and enables a pair of wired or wirelessheadphones to work with the integrated client. The audio content isplayed on the user's headphones. The user is presented with a graphicaluser interface on the integrated client such as those presented in FIGS.3 b-3 i. The user customizes the icons by moving them until he feelscomfortable that he has maximized his ability to hear the audio contentof the movie.

In a related embodiment, the user also has a heads-up display that iswired or communicates wirelessly with the integrated client.Close-captioning text from the movie appears in the user's field ofvision as depicted in FIG. 4 b. The user is presented with a secondgraphical user interface on the integrated client as depicted in FIG. 4c. The user modifies the location, size, color, and transparency of thetext according to his preferences. The user watches the movie with theaudio content and video content customized for his hearing impairment(s)and/or personal preferences.

In yet another embodiment, frequency modulation can be used to adjustthe audio content delivered to the user so as to further improve thehearing ability of a hearing impaired individual. Frequency modulationcan be performed either on the server or client side using a wellresearched set of parameters intended to apply to many hearing impairedindividuals.

The parameters can be designed to reduce the high frequency content ofinterfering noise in all channels of multichannel audio content.Research has shown that high frequency noises cause the mostinterference to hearing. The parameters can be included by the digitalfilters within the controller/receiver or another device capable ofstoring such information. The processor of the controller/receiver canapply the parameters to the multiple channels to provide the desiredfrequency modulation and/or filtering.

In another embodiment, a user can modulate frequencies with a graphicaluser interface that permits a user to increase or decrease the magnitudeof content at a given frequency. In this case, the user interface mightdepict a graphical version of a stereo equalizer. The server or clientproviding the graphical user interface can apply the user frequencypreferences to the audio content to provide the desired frequencymodulation and/or filtering.

The preceding description has been presented with reference to presentlypreferred embodiments of the invention. These should not be construed aslimitations on the scope of the invention, but rather as examples of theembodiments thereof. Alterations and changes in the described structuremay be practiced without meaningfully departing from the scope andspirit of the invention.

1. A system for adjusting audiovisual content comprising: a serverconfigured to: receive a first content from a media source, and generatea second content comprising multiple audio channels based on the firstcontent; and a plurality of clients, where each client is configured to:receive the second content from the server, provide a graphical userinterface to a user on a display, the graphical user interface having aplurality of icons representing the user and the multiple audio channelsat selected positions on the display, wherein a position of any of theplurality of icons is configured to be modified by the user and toaffect a relative volume of the audio channels, generate a third contentbased on the second content in accordance with a relative position ofthe plurality of icons, provide the third content to an audio device,and allow the user to maintain an unbalanced sound condition.
 2. Thesystem of claim 1, wherein each client is further configured to increasea volume of a first audio channel group comprising at least one of themultiple audio channels when a movement of the user icon decreases adistance between a first icon group, comprising the audio iconsrepresenting the audio channels of the first audio channel group, andthe user icon.
 3. The system of claim 2, wherein each client is furtherconfigured to maintain or decrease a volume of the audio channels of thesecond content other than the audio channels of the first audio channelgroup when the movement of the user icon decreases the distance betweenthe first icon group and the user icon.
 4. The system of claim 1,wherein each client is configured to generate the third content suchthat a magnitude of an alteration in second content corresponds to thedistance an icon is moved.
 5. The system of claim 1, wherein thegraphical user interface allows the user to alter the volume of themultiple channels by moving one or more of the plurality of icons. 6.The system of claim 1: wherein the first content comprises videocontent, and wherein each client is configured to provide the videocontent to a video device in synchronization with the third contentprovided to the audio device.
 7. The system of claim 1: wherein thefirst content comprises video content, and wherein each client isconfigured to provide a second graphical user interface to the user,where the second graphical user interface is configured to allow theuser to alter the video content.
 8. The system of claim 7, wherein thesecond graphical user interface is configured to allow the user to altera characteristic of the text, wherein the characteristic is selectedfrom the group consisting of size, color, and translucence.
 9. Thesystem of claim 1: wherein the first content comprises video content,and wherein each client is configured to generate close-captioning textdata based on the second content and provide the close-captioning textdata to a video device.
 10. The system of claim 1: wherein the firstcontent comprises video content, and wherein each client furthercomprises a video device configured to display the video content. 11.The system of claim 10, wherein the video device is a head mounted videodisplay.
 12. The system of claim 11, wherein the head mounted videodisplay is configured to project at least a portion of the video contentdirectly into an eye of the user.
 13. The system of claim 1, wherein atleast one of the clients is a cell-phone or a personal data assistant.14. A method of adjusting audiovisual content comprising: receiving, ata server, a first signal comprising a first content from a media source;generating, at a server, a second content comprising multiple audiochannels based on the first content; providing, at a plurality ofclients, a graphical user interface to a user on a display, thegraphical user interface having an icon arrangement comprising aplurality of icons representing a user and the multiple audio channelsat selected positions on the display, wherein a position of any of theplurality of icons is configured to be modified by the user and toaffect a relative volume of the audio channels; generating, at theplurality of clients, a third signal based on the second content inaccordance with a relative position of the plurality of icons;providing, at the plurality of clients, the third signal to an audiodevice; and allowing, at the plurality of clients, the user to maintainan unbalanced sound condition.
 15. The method of claim 14, furthercomprising: increasing, at the plurality of clients, a volume of a firstaudio channel of the second content when a movement of an iconrepresenting the first audio channel decreases a distance between thefirst audio channel icon and the user icon.
 16. The method of claim 15,further comprising: maintaining or decreasing, at the plurality ofclients, a volume of the audio channels of the second content other thanthe first audio channel when a movement of the first audio channel icondecreases the distance between the first audio
 17. The method of claim14, wherein the generating the third signal based on the second contentand the icon arrangement comprises generating the third signal such thata magnitude of an alteration in second content corresponds to thedistance an icon is moved.
 18. The method of claim 14, wherein thegraphical user interface allows the user to alter the volume of themultiple channels by moving one or more of the plurality of icons. 19.The method of claim 14, wherein the first content comprises videocontent, and wherein each client is configured to provide the videocontent to a video device in synchronization with the third contentprovided to the audio device.
 20. The method of claim 14: wherein thefirst content comprises video content, and wherein each client isconfigured to provide a second graphical user interface to the user,where the second graphical user interface is configured to allow theuser to alter the video content.
 21. The method of claim 20, wherein thesecond graphical user interface is configured to allow the user to altera characteristic of the text, wherein the characteristic is selectedfrom the group consisting of size, color, and translucence.
 22. Themethod of claim 14: wherein the first content comprises video content,and wherein each client is configured to generate close-captioning textdata based on the second content and provide the close-captioning textdata to a video device.
 23. The method of claim 14: wherein the firstcontent comprises video content, and wherein each client furthercomprises a video device configured to display the video content. 24.The method of claim 23, wherein the video device is a head mounted videodisplay.
 25. The method of claim 24, wherein the head mounted videodisplay is configured to project at least a portion of the video contentdirectly into an eye of the user.
 26. The method of claim 11, wherein atleast one of the clients is a cell-phone or a personal data assistant.